Comparison of frequentist and Bayesian inference . Class 20 , 18 . 05 , Spring 2014 Jeremy Orloff and Jonathan Bloom

1 Learning Goals 1. Be able to explain the difference between the p-value and a posterior probability to a doctor. 2 Introduction We have now learned about two schools of statistical inference: Bayesian and frequentist. Both approaches allow one to evaluate evidence about competing hypotheses. In these notes we will review and compare the two approaches, starting from Bayes formula. 3 Bayes formula as touchstone In our first unit (probability) we learned Bayes formula, a perfectly abstract statement about conditional probabilities of events: P (B | A)P (A) P (A | B) =. P (B) We began our second unit (Bayesian inference) by reinterpreting the events in Bayes formula: P (D | H)P (H) P (H | D) =. P (D) Now H is a hypothesis and D is data which may give evidence for or against H. Each term in Bayes formula has a name and a role. • The prior P (H) is the probability that H is true before the data is considered. • The posterior P (H | D) is the probability that H is true after the data is considered. • The likelihood P (D | H) is the evidence about H provided by the data D. • P (D) is the total probability of the data taking into account all possible hypotheses. If the prior and likelihood are known for all hypotheses, then Bayes formula computes the posterior exactly. Such was the case when we rolled a die randomly selected from a cup whose contents you knew. We call this the deductive logic of probability theory, and it gives a direct way to compare hypotheses, draw conclusions, and make decisions. In most experiments, the prior probabilities on hypotheses are not known. In this case, our recourse is the art of statistical inference: we either make up a prior (Bayesian) or do our best using only the likelihood (frequentist).